Incremental view maintenance and data lineage tracing in heterogeneous database environments

With the increasing amount and diversity of information available on the Internet, there has been a huge growth in information systems that need to integrate data from distributed, heterogeneous data sources. Automed (Automatic Generation of Mediator Tools for Heterogeneous database Integration) is a database transformation and integration system, which is designed to support virtual and materialized integration of schemas expressed in a variety of modelling languages. In previous work of the Automed project [21, 16, 17], a general framework has been developed to support schema transformation and integration in heterogeneous database architectures. The framework consists of a low-level hypergraph based data model (HDM) and a set of primitive schema transformations on HDM schemas. We term the sequence of primitive transformations defined for transforming a schema S1 to a schema S2 a transformation pathway from S1 to S2. That is, a transformation pathway consists of a sequence of primitive transformations. The purpose of my research is to investigate techniques for incremental view maintenance and data lineage tracing for integrated databases which have been formed from heterogeneous source databases via Automed schema transformation pathways (data lineage tracing investigates how data in a data warehouse has been derived from the data sources). My approach is to decompose the processes of incremental view maintenance and data lineage tracing into a sequence of simple steps based on the transformation pathways. I use a functional intermediate query language (IQL) as the query language to implement the algorithms for these two aspects. The remainder of this paper is as follows. Section 2 outlines the related work. Section 3 gives the examples of the IQL language and Automed transformation pathways. Section 4 presents my research questions and research approach. The preliminary ideas and results achieved so far are given in Section 5. Section 6 describes contributions of my research so far and directions of future work.